The present invention relates to an image processing apparatus and method thereof. More particularly, this invention relates to an image processing apparatus and method thereof which can interpolate the image signals of missing pixels.
When a single-plate color CCD on which color filters of RGB as shown in FIG. 4 are arranged in a checkered pattern (RGB are arranged in every alternate pixel in the horizontal and vertical directions) is used, each of the pixels can have only one signal component out of the colors RGB. In such a case the remaining two color components are required to be obtained by interpolation.
In the conventional interpolation missing pixels (meaning pixels with missing values) are obtained simply by using an average of pixels around each of the missing pixels. More specifically, assuming that each pixel of the color CCD is represented by (j, i) as shown in FIG. 5, interpolation is performed through computations 1 to 4 described below.
1) Interpolation at a position of G (i, j)
R=(R (i, jxe2x88x921)+R (i, j+1))/2
G=G (i, j)
B=(B (ixe2x88x921, j)+B (i+1, j))/2
2) Interpolation at a position of B (i+1, j)
3) Interpolation at a position of R (i, j+1)
4) Interpolation at a position of G (i+1, j+1)
R=((R (i, j+1)+R (i+2, j+1))/2
G=G (i+1, j+1)
B=(B (i+1, j)+B (i+1, j+2))/2
However, in the above mentioned method of interpolating the missing pixels each simply by using an average of pixels around each of the missing pixels, when resolution is less or when dense lines exist within an image, jaggies called as a zipper effect occur, which causes image quality to be degraded. Especially, since a G signal contributes the most to a brightness component, image quality (especially resolution) is largely affected by the fact whether interpolation for the G signal is properly performed or not.
As a technology to solve the problems, there has been known, for instance, a pixel interpolating device in Japanese Patent Laid-Open Publication No. HEI 5-75861.
This pixel interpolating device comprises a binary coding unit for binary-coding each data of the four peripheral pixels located at positions surrounding each of the missing pixels using a prespecified threshold value as a reference. Further, a pixel selecting unit selects the data of two or four peripheral pixels used for computing interpolation data from all combinations of data for the peripheral pixels according to the situation of the binary-coded data for the peripheral pixels. An average value of the peripheral pixel data selected by the pixel selecting unit is determined as the interpolation data of the missing pixel. Namely, average of each data of two or four pixels is determined as the interpolation data according to patterns of each binary-coded data for the four peripheral pixels.
However, the pixel interpolating device in the Japanese Patent Laid-Open Publication No. HEI 5-75861 has a following problems. Namely, when a signal has a low signal level it is largely affected by noise so that correct determination can not be made only with four peripheral pixels. Further, because the binary coding is performed, incorrect determination may cause an image to disadvantageously be more degraded than that in the conventional type of interpolating method.
This invention has been made in the light of the above problems. It is an object of the present invention to provide, an image processing apparatus and method thereof enabling reproduction of an image with high resolution and smooth outline when interpolation for an image signal is needed.
According to a first aspect of this invention, interpolation signals of a pixel to be interpolated are generated according to each degree of changes in image signals of four adjacent pixels located at positions surrounding the pixel to be interpolated and to each degree of changes in image signals of the pixel to be interpolated. Thus, weighting for pixel interpolation can continuously be changed according to the each degree of changes in image signals of the pixel to be interpolated and each degree of changes in image signals of pixels adjacent to the pixel to be interpolated.
According to a second aspect of this invention, interpolation signals of a pixel to be interpolated are generated according to each degree of changes in image signals of the pixel to be interpolated and a minimum values of each degree of changes in four adjacent pixels located at positions surrounding the pixel to be interpolated in the horizontal direction and vertical direction.
According to a third aspect of this invention, interpolation signals of a pixel to be interpolated are generated according to each degree of changes in image signals of two adjacent pixels located on the two sides of the pixel to be interpolated, each degree of changes in image signals of the pixel to be interpolated, and a difference in degrees between an image signal of each of the adjacent pixels to the pixel to be interpolated in the horizontal direction and each image signal of pixels around the adjacent pixel to the pixel to be interpolated in the horizontal direction.
According to a fourth aspect of this invention, a first computing unit computes each degree of changes in the horizontal direction and vertical direction of image signals of four adjacent pixels located at positions surrounding a pixel to be interpolated, a second computing unit computes each degree of changes in the horizontal direction and vertical direction of image signals of the pixel to be interpolated, a first minimum-value computing unit computes a minimum value of the each degree of changes in the horizontal direction of image signals of the four adjacent pixels located at positions surrounding the pixel to be interpolated computed by the first computing unit, a second minimum-value computing unit computes a minimum value of the each degree of changes in the vertical direction of image signals of the four adjacent pixels located at positions surrounding the pixel to be interpolated computed by the first computing unit, and a interpolation-signal computing unit generates interpolation signals of the pixel to be interpolated each by weighting and averaging pixels in the horizontal direction and pixels in the vertical direction of the four adjacent pixels located at positions surrounding the pixel to be interpolated according to a result of computation in the second computing unit and the minimum values computed by the first and second minimum-value computing unit. Therefore, weighting for pixel interpolation can continuously be changed according to each degree of changes in a pixel to be interpolated as well as to each degree of changes in pixels adjacent to the pixel to be interpolated.
According to a fifth aspect of this invention, a first computing unit computes each degree of changes in the horizontal direction and vertical direction of image signals of two adjacent pixels located on the two sides of a pixel to be interpolated, a second computing unit computes each degree of changes in the horizontal direction and vertical direction of image signals of the pixel to be interpolated, a difference determining unit determines a difference in degrees between an image signal of an adjacent pixel to the pixel to be interpolated in the horizontal direction and each image signal of pixels around the adjacent pixel to the pixel to be interpolated in the horizontal direction, a first minimum-value computing unit computes a minimum value of the each degree of changes in the horizontal direction of image signals of the two adjacent pixels located on the two sides of the pixel to be interpolated computed by the first computing unit, a second minimum-value computing unit computes a minimum value of the each degree of changes in the vertical direction of image signals of the two adjacent pixels located on the two sides of the pixel to be interpolated computed by the first computing unit, a first comparing unit compares the minimum value computed by the first minimum-value computing unit with a first threshold value, a second comparing unit compares the minimum value computed by the second minimum-value computing unit with the first threshold value, a third comparing unit compares each degree of changes in the horizontal direction of image signals of the pixel to be interpolated computed by the second computing unit with a second threshold value, a fourth comparing unit compares each degree of changes in the vertical direction of image signals of the pixel to be interpolated computed by the second computing unit with the second threshold value, a selecting unit selects interpolation constants according to results of comparison in the first to fourth comparing units as well as to a result of determination in the difference determining unit, and a interpolation-signal computing unit generates interpolation signals of the pixel to be interpolated each by weighting and averaging pixels in the horizontal direction and pixels in the vertical direction of the four adjacent pixels located at positions surrounding the pixel to be interpolated according to the interpolation constants selected in the selecting unit.